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TITLE: WEB BROWSER WITH ANNOTATION CAPABILITY 



FIELD OF THE INVENTION 

The present invention relates to a web browser with annotation capability. 

5 BACKGROUND OF THE INVENTION 

Currently, e-mail is the dominant vehicle used in time shifted (as opposed to 

on-line) document collaboration between multiple parties using the Internet as the 
transport. The reason e-mail is being used is simply that it has evolved from its use 
internally in organizations to its use across organizations (the network has simply 
10 been extended and the tool has remained). However, the use of e-mail as a tool for 
collaboration between parties has pitfalls: 

1 . The volume of e-mail makes it impossible to effectively manage content. The 
communication thread and the association to specific subject matter become 
very obscure or non-existent. Too many copies of the document (subject 

1 5 matter) make it difficult to achieve a common conclusion based on input from 

multiple parties. Collaboration is accomplished via the exchange and 
modification of documents and this in the format of the production software 
(i.e. word processor), creating numerous compatibility and version issues. 

2. The e-mail repository is personal meaning the documents cannot easily be 
20 shared, without extensive transport, thereby once again creating versioning 

issues. 

3. Versioning and document comparison issues result because of exchange of 
documents via the production tool, (i.e. the word processors). 

25 Another approach to achieving on-line collaboration is based on the web 

technology. The document on which users collaborate is made available on a network 
server. Remote access to the document is possible through a web browser at a 
remotely located client. The server sends a file containing the document expressed in 
Hyper Text Markup Language (HTML) to the web browser that renders the document 

30 on the display such that the user can see it. 

The use of a web browser to render the document largely resolves the 
compatibility issues arising when the document is distributed in the format of the 
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production software. The HTML language in which the document is sent to the web 
browser is a standard and can be interpreted by any browser, independent of the 
browser vendor. 

5 Currently available web browsers do not provide effective annotation tools. 

While a user can view a document, the process for adding comments to the document 
and making those comments available to the author or to other users is inexistent or 
cumbersome at best. To alleviate this difficulty, it has been suggested to provide 
extensions to the basic browser such as to allow a reasonable level of annotation 
10 functionality. These extensions include the use of new HTML annotations tags, the 
use of Extended Markup Language (XML), or creating sophisticated add-ins such as 
plug-ins or java applets. 

Although the add-ins approach can provide effective annotation functionality, 
15 it raises compatibility issues since all users in the group of collaborators must have 
compatible add-ins. 

Against this background, it clearly appears that a need exists in the industry to 
develop annotation tools for a web browser without the need of add-ins. 

20 SUMMARY OF THE INVENTION 

In a broad aspect, the invention provides a file suitable for processing by a 

web browser executing on a computer having a display. The file includes a first set of 
data causing the browser to render on the display a fixed representation of a document 
by implementing a document object model. The document object model has one or 
25 more methods to manipulate HTML. The file also has a second set of data donating to 
the web browser an annotation functionality permitting a user to annotate the fixed 
representation of the document. The annotation is implemented by invocation of one 
or more methods of the document object model that manipulates HTML. 



30 



In this specification, "manipulating HTML" means inserting, removing or 
modifying HTML. 
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This approach donates to the web browser effective annotation functionality 
without the necessity of traditional add-ins. Since the annotation is made on a fixed 
representation of the document, the annotation data can be positively located in the 
document, with little or no ambiguity. 

In this specification the expression "file" should be interpreted as a collection 
of information containing user data, such as fixed representation of the document, 
program data, or any other information used by a program or necessary for the 
execution of a program, without limitation on the particular format in which the 
collection information is expressed, transmitted or stored. 

In this specification, the expression "fixed representation of a document" is a 
representation where the position of the various elements of the image, containing 
text, non-text information or both, one with respect to another, stay fixed, 
independently of the particular web browser on which the representation is rendered. 
For instance, in a multi-page document, each page will contain exactly the same text 
and specific attributes such as footers, headers, indents, text wrapping around images 
and image absolute positioning will be the same independently of the browser 
rendering the page. One example of a fixed representation is a representation of a 
document in printed format. 

A fixed representation of a document can be electronically expressed, 
transmitted or stored in formats such as Graphics Interchange Format (GIF), Joint 
Photographic Experts Group (JPEG) format, Tagged Image File Format (TIFF) and 
Bitmap (BMP) format, among others. This list should not be considered exhaustive 
as other possibilities exist without departing from the spirit of the invention. 

In a specific and non-limiting example of implementation, the data file is 
transmitted from a server to a client machine on which the web browser is executing. 
The file is stored in a computer readable storage medium in the client machine, such 
as a rigid drive storage unit or the Random Access Memory of the machine. The file 
is loaded by the web browser. The web browser renders the fixed representation of 
the document. The second set of data in the file is interpreted to create at least one 
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annotation tool that the user can invoke to annotate the fixed representation of the 
document. The annotation tool can be in the form of a highlight tool, a pen tool and a 
notes tool, among others. The tool is accessed by pressing a button on a toolbar, by 
pressing a key on the keyboard, or any other suitable invocation procedure. 

5 

The operation of the annotation tool creates annotation data. The annotation 
data is HTML data, appended to the original data file in the form of a layer to the 
fixed representation of the document. The annotation data includes an annotation 
image element and coordinates information. The annotation image element includes 

10 the information added by the user. For instance, in the case of the pen tool, the user 
circles words, phrases or paragraphs to highlight them. The annotation image element 
is the trace of the pen. The annotation image element can be stored in the original file 
in formats such as Graphics Interchange Format (GIF), Joint Photographic Experts 
Group (JPEG) format, Tagged Image File Format (TIFF) and Bitmap (BMP) format, 

15 among others. 

The coordinated information includes data specifying the position of the 
annotation image element in the fixed representation of the document. This allows to 
properly registering the annotation image element with a component of the document 
20 that is being annotated. 

After the annotation procedure is completed, the user has the option of 
returning the original data file to which has been appended the annotation data to the 
source such that it can be stored there for future reference. 

25 

The invention also provides a method for annotating a document, including 
processing a file by a web browser executing on a computer having a display. The file 
includes a first set of data causing the browser to render on the display a fixed 
representation of a document by implementing a document object model that has at 
30 least one method for manipulating HTML. The file also includes a second set of data 
donating to the web browser an annotation functionality permitting a user to annotate 
the fixed representation of the document by invocation of at least one method of the 
document object model that manipulates HTML. 
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The invention further provides a signal transmission medium containing a data 
signal representing a file suitable for processing by a web browser executing on a 
computer having a display. The file has a first set of data causing the browser to 

5 render on the display a fixed representation of a document by implementing a 
document object model that has at least one method for manipulating HTML, The 
file also includes a second set of data donating to the web browser an annotation 
functionality permitting a user to annotate the fixed representation of the document by 
manipulating HTML by invocation of at least one method of the document object 

10 model. 

BRIEF DESCRIPTION OF THE DRA WINGS 

A detailed description of examples of implementation of the present invention 

is provided herein below with reference to the following drawings, in which: 

15 

Figure 1 is a diagram of a network based collaborative system where a 
plurality of workstations communicate with a server on which are stored files to be 
accessed by the individual workstations; 

20 Figure 2 is a block diagram of a workstation shown in Figure 1; 

Figure 3 illustrates the structure of a file stored on the server and that can be 
accessed by the workstations; 

25 Figure 4 is a functional block diagram of a web browser executing on a 

workstation; 

Figure 5 is an example of a rendering on the display of the workstation of 
Figure 4 of a fixed representation of a document; 

30 

Figure 6 illustrates the image on the display after the user has applied an 
annotation; 



86052-3 



Page 6 



July 31, 2001 



Figure 7 illustrates a conceptual representation of the annotation made in 
Figure 6, in the form of a layer to the fixed representation of the document; and 

5 Figure 8 illustrates the structure of the file shown in Figure 3, to which has 

been appended annotation data. 

In the drawings, embodiments of the invention are illustrated by way of 
example. It is to be expressly understood that the description and drawings are only 
10 for purposes of illustration and as an aid to understanding, and are not intended to be a 
definition of the limits of the invention. 



DETAILED DESCRIPTION 

Figure 1 shows a simplified diagram of a system of interconnected computers 

15 over which users can collaborate. The system includes a server 10 that connects to 
individual workstations 12, 14, 16 and 18 over a network 20 such as the Internet. The 
server 10 holds resources that can be accessed through the individual workstations 12, 
14, 16 and 18. 

20 Figure 2 is a block diagram of the workstation 12. The structure and operation 

of the other workstations 14, 16 and 18 are identical and will not be described in 
detail here. 

The workstation 12 includes a Central Processing Unit (CPU) 22 connected to 
25 a storage medium 24 over a data bus 26. Although the storage medium 24 is shown 
as a single block, it may include a plurality of separate components, such as a floppy 
disk drive, a fixed disk and Random Access Memory (RAM), among others. The 
workstation also includes an Input/Output interface 28 that connects to the data bus 
26. The I/O 28 is the entity through which the workstation 12 communicates with the 
30 external world. For instance, data sent from or received by the workstation 12, during 
a communication session with the server 10 or any other workstation 14, 16 and 18 
transits through the I/O 28. 
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A monitor 30, connecting to the data bus 26, displays information to the user. 
The user can enter commands through a keyboard 32 and a pointing device in the 
form of a mouse 34. 

During collaboration, the user at the workstation 12 connects with the server 
10 to access a file residing on the server 10. The data exchange is made by using a 
protocol such as the Transport Control Protocol/Internet Protocol (TCP/IP). The user 
communicates with the server 10 through a web browser executing on the workstation 
12. The web browser is a program element including individual instructions and 
residing in the storage medium 24. The individual instructions are executed by the 
CPU 22 to implement the browser functionality. A functional block diagram of the 
web browser is shown at Figure 4. The web browser 36 includes a Graphical User 
Interface 38 and a central manager 40 that manages communications with external 
entities, such as the server 10, performs the necessary operations to allow the user to 
view through the GUI 38 Hypertext Markup Language (HTML) documents, etc. It is 
not deemed necessary here to discuss in detail the structure and operation of the GUI 
38 and the manager 40, since these components are off the shelf items. 

When the communication session with the server 10 is established, the user 
identifies through the web browser 36 the file to be accessed. The server 10 sends the 
file to the workstation 12 over the signal transmission medium linking both entities. 
The file is encoded according to the selected protocol, such as TCP/IP and forwarded 
to the workstation. 

Figure 3 shows the data structure of the file. The file has two components, 
namely a first set of data 42 and a second set of data 44. The first set of data is user 
information and contains a fixed representation of a document. The document may be 
a single page document or a multiple page document. In one specific example, the 
fixed representation is a representation of the document in printed form. This 
representation may be expressed by encoding the image of the printed document in a 
format such as Graphics Interchange Format (GIF), Joint Photographic Experts 
Group (JPEG) format, Tagged Image File Format (TIFF) and Bitmap (BMP) format, 
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among others. The second set of data is program data that allows the web browser to 
implement an annotation functionality. 



When the data representing the file is received by the workstation 12 it is 
5 placed in the storage medium 24 and then loaded in the web browser 36. The web 
browser 36 interprets the first set of data and renders the fixed representation of the 
document by implementing a document object model. Assume that the document is a 
single page document and that it is a non-text document, consisting of the image of 
two sheets of paper. The image displayed by the monitor is shown at Figure 5. 

10 

The web browser 36 also interprets the second set of data, which is program 
data. The program data implements in the web browser 36 at least one annotation tool 
52 3 shown at Figure 4. The annotation tool 42 is made available to the user through 
the GUI 38. Possible examples of this implementation include creating on the 
15 existing toolbar a series of buttons (not shown), each associated with a certain 
annotation tools that are independently selectable. Examples of annotation tools 
include a pen tool, a notes tool and a highlight tool, among others. 

When a user selects an annotation tool by clicking on the button associated 
20 with that tool, it invokes a method of the document object model to manipulate 
HTML. Examples of such methods in Dynamic Hypertext Markup Language 
(DHTML) include: 

1 . ApendChild method - Appends an element as a child to an object; 
25 2. Add method - Adds an element to the: 

a. areas. asp - areas; 

b. controlRange.asp - Control Range; 

c . options .asp - options collections ; 

3. AppendData method - Adds a new character string to the end of the 
30 object; 

4. CloneNode method - Copies a reference to the object from the 
document hierarchy; 
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5. CreateCaption method - Creates a caption; 

6. CreateComment method - Creates a comment; 

7. CreateControlRange method - Creates a control range collection of 
nontext elements; 

5 8. CreateElement method - Creates an instance of the element for the 

specified tag; 

9. CreateTextNode method - Creates a text string from the specified 
value; 

10. CreateTextRange method - Creates a text range object for the 
10 element; 

1 1 . DeleteCell method - Remove a specified cell; 

12. DeleteData method - Removes a specified range of characters 
from the object; 

1 3 . Duplicate method - Returns a duplicate of the text range; 
15 14. Get Adj acentText method - Returns the adj acent text string; 

15. Insert Adj acentElement method — Inserts an element at the 
specified location; 

16. InstertsAdjacentHTML method - Inserts the given HTML text 
into the element at the location; 

20 17. Write method - Writes one or more HTML expressions to a 

document in the specified window; 

18. X method - Sets or retrieves the x-coordinate, in pixels, of the 
mouse pointer's position relative to the parent element; 

19. Y method - Sets or retrieves the y-coordinate, in pixels, of the 
25 mouse pointer's position relative to the parent element; 

It is expressly noted that the above methods are merely examples. Other 
methods can be employed without departing from the invention. 



30 A detailed example is shown in Figure 6. In this example, the pen annotation 

tool is selected. When the user is ready to make the annotation, a button on the mouse 
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34 is depressed. The button click invokes a method of the document object model 
that starts generating HTML annotation data. The annotation data includes two 
components. The first component is an annotation image element, which describes 
the trace followed by the mouse pointer 54 on the monitor 30. In one example, the 

5 annotation image element is expressed in a format such as Graphics Interchange 
Format (GIF), Joint Photographic Experts Group (JPEG) format, Tagged Image File 
Format (TIFF) and Bitmap (BMP) format, among others. The second component of 
the annotation data is to coordinate information specifying the position of the 
annotation image element in the fixed representation of the document. The 

10 coordinates information can be the coordinates of the mouse pointer 54 when the 
annotation started. 

The annotation tool 52 generates the annotation data as HTML language, 
which is appended to the original file and forms a layer to the fixed representation of 
15 the document. This layer is conceptually shown at Figure 7. The annotation image 
element is separate from the fixed representation of the document but it can be 
superposed to form a compound image to show where the annotations are made. 

In light of the fact that the document is expressed in a fixed representation 
20 format, the registration between the annotation image element in the layer and the 
component of the document being annotated can be accurately maintained. When the 
file including the annotation data is rendered on a browser different than the browser 
used to create the annotation, this registration will be maintained, thus the annotation 
will be in its proper position. 

25 

The annotation data can also include additional information such as the 
identity of the author that has made the annotation, a date and time stamp as well as 
any other suitable information. This additional information is expressed in HTML 
language as well. 

30 

The annotation data is appended to the original file as shown at Figure 8. As 
mentioned previously, it is maintained in the form of a layer separate from the 
original fixed representation of the document. 
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For the purpose of this specification, it is not deemed necessary to provide a 
detailed description of the notion of layers, since storing, creating or manipulating 
images as layers is something well known in the art. 

Although various embodiments have been illustrated, this was for the purpose 
of describing, but not limiting, the invention. Various modifications will become 
apparent to those skilled in the art and are within the scope of this invention, which is 
defined more particularly by the attached claims. 



